Symbol Grounding for Robot Dialog Research Proposal

Advancements in robotics have led to an ever-growing repertoire of software capabilities (e.g., recognition, mapping, and object manipulation). However, robotic capabilities grow, the complexity of operating and interacting with such robots increases (such as through speech, gesture, scripting, or programming). Language-based communication can offer users the ability to work with physically and computationally complex robots without diminishing the robot’s inherent capability. However, it remains an open question how to build a common ground in human language that will scale with the growth of robot capabilities, for instance within development environments such as ROS (the Robot Operating System). We examine this scaling problem through large-scale symbol grounding for robot dialog. We explore this problem in two parts through the development of: (1) a generic software framework for ROS that grounds parts of speech (verbs for now) into robotic capabilities using the proposed action hierarchy model and (2) a dialog interface for human-robot interaction through an expressive subset of natural language. We will evaluate the framework and interface through mobile manipulation experiments with a PR2, with consideration of the future scalability of our approach. Background and Related Work [Winograd, 1971] developed SHRDLU, a system which processed natural-language instructions and performed actions in a virtual environment. From this, researchers pushed forward trying to extend SHRDLU’s capabilities into real-world environments and soon branched into tackling various subproblems, including NLP and robotics systems. Research conducted on the robotics systems side has resulted in frameworks such as ROS, developed by [Quigley et al., 2009], which has been used in several domains of modern robotics research. NLP research including robotic components has also lead to advancements. Notably, [Kollar et al., 2010] and [MacMahon et al., 2006] have developed methods of following route instructions given in natural language. [Dzifcak et al., 2009] studied translating natural language instructions into goal descriptions and actions. [Chernova et al., 2010] implemented natural-language and action-oriented human-robot interaction with humans in a task by data-mining previous human-human interactions of the same task. However, the scalability of these solutions outside of their test domain remains open. Attempts have been made recombining these fields. For instance, [Tenorth et al., 2010] has developed robotic systems capable of inferring and acting upon implicit commands using knowledge databases.